A so-called omnidirectional camera is cited in Japanese Patent No. 2939087 as a conventional example of a wide-angle imaging device that uses a reflecting plane. This is a device capable of forming an image on an image plane in 360 degrees of direction at once around a rotational axis, by means of a reflecting plane having a shape that is rotary-symmetric around the optical axis of the camera. This device also produces a so-called panoramic image by performing image conversion processing on the omnidirectional image thus picked up.
The omnidirectional camera is effective when the monitoring area includes a 360-degree periphery. However, when a monitoring area of about 180 degrees is sufficient, the omnidirectional camera picks up an image with about 180 degrees of unnecessary area, which is an inefficient use of the imaging element.
A wide-angle imaging device is therefore conceived that would pick up an image within the reflected image made by the reflecting plane that includes only about ±90 degrees of a referemce direction, which is any direction perpendicular to the central rotation axis of the reflecting plane.
A developed image, which is a type of image composed of a connected string of images picked up with a camera of the usual angle of view, can also be obtained by performing computational processing on an image obtained with this type of wide-angle imaging device (hereinafter referred to as “wide-angle image”), in the same manner as with the omnidirectional camera disclosed in Japanese Patent No. 2939087.
Development processing is described hereinafter.
FIG. 33 schematically depicts a wide-angle imaging device 8 to describe development processing. The coordinate axes are defined as follows: the direction away from the paper surface toward the viewer is the X-axis, the left-right direction of the paper surface is the Y-axis, and the up-down direction of the paper surface is the Z-axis. The Z-axis also conforms to the vertical direction.
The wide-angle imaging device 8 is composed of a hyperbolic reflecting mirror 1 having a rotary-symmetric shape, an image-forming lens 2, and an imaging element 3. An imaging device is normally composed of the image-forming lens 2 and the imaging element 3.
The hyperbolic reflecting mirror 1 has a reflecting plane that covers an area of ±90 degrees about the central rotational axis of the hyperbolic plane. The 0-degree direction thereof is the direction of the reference optical axis, which is in the same direction as the Y-axis. 4 is the internal focal point of the hyperbolic reflecting mirror 1. 6 is the central rotational axis of the hyperbolic reflecting mirror 1, and conforms to the Z-axis.
A light beam that is emitted from an arbitrary point P(X, Y, Z) in space and is incident on the internal focal point 4 is reflected off the reflecting plane of the hyperbolic reflecting mirror 1, and condensed to an external focal point (not pictured) on the central rotational axis 6. The position of the external focal point and the position of the principal point 7 of the image-forming lens 2 are disposed so as to substantially conform to each other, and the light beam forms an image on the imaging plane 5 of the imaging element 3 via the image-forming lens 2.
Also, 21 is a hypothetical cylindrical plane, which is a hypothetical projection plane used for mapping from a wide-angle image to a developed image. The central rotational axis of the hypothetical cylindrical plane 21 conforms to the central rotational axis 6 of the hyperbolic reflecting mirror 1.
A point (specifically, a point on the wide-angle image) p(x, y) corresponding to the arbitrary point P(X, Y, Z) in space is defined on the imaging plane 5. The hyperbolic shape of the hyperbolic reflecting mirror 1 is defined by the following formula (Eq. 1).(X2+Y2)/a2−Z2/b2=−1  (Eq. 1)Wherein a and b are constants for determining the shape of the hyperboloid. Also,c=(a2+b2)0.5  (Eq. 2)Eqs. 3 through 5 are also established:Z=(X2+Y2)0.5·tan β+c  (Eq. 3)tan β={(b2+c2)·sin α−2bc}/{(b2−c2)·cos α}  (Eq. 4)tan α=F/(x2+y2)0.5  (Eq. 5)
Here, F is the focal distance of the imaging device that comprises the image-forming lens 2 and the imaging element 3. Angles α and β are as referenced in FIG. 33.
If X, Y, and Z, and b, c, and F are defined by Eq. 3 through Eq. 5, then (x2+y2)0.5; specifically, the distance from the point of intersection with the central rotational axis 6 on the imaging plane 5 to point p is defined.
Also, because a light beam directed towards the internal focal point 4 reflects off the hyperbolic reflecting mirror 1 towards the external focal point in hyperbolic fashion, the respective directions of the points P and p in the XY plane and xy plane conform to each other. Consequently,Y/X=y/x  (Eq. 6)
If X and Y are obtained from Eq. 6, the direction of point p is also defined, and a unique point (specifically, a point on the wide-angle image) p(x, y) corresponding to P(X, Y, Z) can therefore be defined on the imaging plane 5. Specifically, a developed image can also be generated in a wide-angle imaging device, such as one designed for picking up an image with a substantially ±90-degree field of view, by defining the projection plane (hypothetical cylindrical plane 21) of the developed image, determining the corresponding point of each of the pixels of the developed image on the wide-angle image by taking into account the mapping of the wide-angle image to the projection plane, and determining the luminance values of all the pixels in the developed image accordingly, as in the case of the omnidirectional camera.
However, if the central rotational axis of the reflecting plane in this case is tilted from the vertical direction in a plane perpendicular to the reference direction, then ignoring this tilt, performing computational processing for the image obtained from this wide-angle imaging device, and converting a substantially ±90-degree wide-angle image to a developed image will make the outer rim of the developed image no different from when there is no tilt in the central rotational axis of the reflecting plane. The contents of the developed image will acquire a tilt commensurate with the inclination of the central rotational axis of the reflecting plane, and will produce the drawback of giving a sense of discomfort to the viewer.
Also, the field of view of the wide-angle imaging device in the direction of the central rotational axis generally satisfies the condition |β|<|γ|, where β is the elevation angle and γ is the depression angle (see FIG. 33). Because of this, the position within the developed image for which the elevation angle is zero (hereinafter referred to as “line-of-sight center”) shifts towards either side of the developed image. A drawback arising this time is that portions within the developed image that are distant from the line-of-sight center look more slanted than other portions, and hence look distorted in comparison to the other portions.
The omnidirectional camera will also be described using FIG. 34.
FIG. 34(a) depicts the basic structure of the omnidirectional camera. The omnidirectional camera 100 comprises a hyperbolic reflecting mirror 101 as a reflecting plane having a rotary-symmetric shape, an image-forming lens 102, and a CCD or other imaging element 103.
104 is the internal focal point of the hyperbolic reflecting mirror 101. 105 is the imaging plane of the imaging element 103. 106 is the central rotational axis of the hyperbolic reflecting mirror 101, and conforms to the Z-axis in this case. Also, an arbitrary direction perpendicular to the central rotational axis of this hyperbolic reflecting mirror 101 is designated as the Y-axis direction. 107 is the principal point of the image-forming lens 102. The origin O of the coordinate system is at the point that divides a straight line connecting the internal focal point 104 with the principal point 107 into two equal parts.
Also, 108a and 108b are light beams incident on the hyperbolic reflecting mirror 101, and 109a and 109b are the incident points of light beams 108a and 108b on the hyperbolic reflecting mirror 101. 110a and 110b are the points at which the light beams 108a and 108b reflected at the incidence points 109a and 109b form an image in the imaging plane 105.
The light beams 108a and 108b, which are emitted from points in the YZ plane, propagated towards the internal focal point 104, and cast on the incident points 109a and 109b on the hyperbolic reflecting mirror 101, are reflected by the reflecting plane of the hyperbolic reflecting mirror 101, and are condensed to an external focal point (not pictured) on the central rotational axis 106. The position of this external focal point and the position of the principal point 107 of the image-forming lens 102 are disposed so as to substantially conform to each other, and the light beams 108a and 108b form an image at the image formation points 110a and 110b on the imaging plane 105 of the imaging element 103 via the image-forming lens 102.
FIG. 34(b) depicts the manner in which an image is formed on the imaging plane 105. As depicted in the diagram, the Y-axis is directed upwards on the paper surface. Also, the Z-axis is directed perpendicular to the paper surface. Luminous fluxes (not pictured), which include the light beams 108a and 108b, and which are incident on the hyperbolic reflecting mirror 101 towards the internal focal point 104 from 360 degrees about the central rotational axis 106 of the omnidirectional camera 100, pass through the hyperbolic reflecting mirror 101 (not pictured) and form a circular image 111 on the imaging plane 105 via the image-forming lens 102, in the same manner as the light beams 108a and 108b. Both ends of the Y direction also become the image formation points 110a and 110b of the light beams 108a and 108b in the circular image 111.
Development processing for converting a circular omnidirectional image obtained as described above into a panoramic image will be described hereafter.
FIG. 35 schematically depicts the omnidirectional camera 100 in order to describe development processing. The hyperbolic reflecting mirror 101, image-forming lens 102, imaging element 103, internal focal point 104, imaging plane 105, central rotational axis 106, and principal point 107 are the same as described in FIG. 34.
112 is a hypothetical cylindrical plane, which is a hypothetical projection plane used for mapping from a circular omnidirectional image to a developed image. The central rotational axis of the hypothetical cylindrical plane 112 conforms to the central rotational axis 106 of the hyperbolic reflecting mirror 101. The coordinate axes are defined as follows: the direction away from the paper surface toward the viewer is the X-axis, the left-right direction of the paper surface is the Y-axis, and the up-down direction of the paper surface is the Z-axis. The Z-axis is also defined as conforming to the central rotational axis 106.
As previously described, a light beam that is emitted from an arbitrary point P(XP, YP, ZP) in space and is incident on the internal focal point 104 is reflected off the reflecting plane of the hyperbolic reflecting mirror 101, and is condensed to an external focal point (not pictured) on the central rotational axis 106. The position of the external focal point and the position of the principal point 107 of the image-forming lens 102 are disposed so as to substantially conform to each other, and a light beam forms an image on the imaging plane 105 of the imaging element 103 via the image-forming lens 102.
A point on the imaging plane 105 that corresponds to the arbitrary point P(XP, YP, ZP) in space, that is, p(xp, yp), is defined as a point on a circular omnidirectional image. The hyperbolic shape of the hyperbolic reflecting mirror 101 is defined by Eq. 7.(X2+Y2)/a2−Z2/b2=−1  (Eq. 7)Here, a and b are constants for determining the shape of the hyperboloid. Also,c=(a2+b2)0.5  (Eq. 8)Eq. 9 through Eq. 11 are also established:Z=(X2+Y2)0.5·tan α+c  (Eq. 9)tan α={(b2+c2)·sin β−2bc}/{(b2−c2)·cos β}  (Eq. 10)tan β=F/(x2+y2)0.5  (Eq. 11)
Here, F is the focal distance of the imaging device that comprises the image-forming lens 102 and the imaging element 103. Angle α is the angle formed between the XY plane and the straight line that connects the internal focal point 104 with the arbitrary point P. Angle β is the angle formed between the XY plane and the straight line that connects the principal point 107 (which substantially conforms to the external focal point of the hyperbolic reflecting mirror 101) with the point p on the imaging plane 105. If XP, YP, and ZP, and b, c, and F are defined by Eq. 9 through Eq. 11, then (xp2+yp2)0.5; specifically, the distance from the point of intersection with the central rotational axis 106 on the imaging plane 105 to point p is defined.
Also, because a light beam directed towards the internal focal point 104 reflects off the hyperbolic reflecting mirror 101 towards the external focal point in hyperbolic fashion, the respective directions of the points P and p in the XY plane and xy plane conform to each other. Consequently,Y/X=y/x  (Eq. 12)
If XP and YP are defined by Eq. 12, the direction of point p is also defined.
As described above, a unique point (specifically, a point on the circular omnidirectional image) p(xp, yp) corresponding to P(XP, YP, ZP) can be defined on the imaging plane 5. Consequently, a developed image can be generated by defining the projection plane (hypothetical cylindrical plane 112) of the developed image, determining the corresponding point of each of the pixels of the developed image on the circular omnidirectional image by taking into account the mapping of the circular omnidirectional image to the projection plane, and determining the luminance values of all the pixels in the developed image accordingly.
The omnidirectional camera is effective when the monitoring area includes a 360-degree periphery. However, when a monitoring area of about 180 degrees is sufficient, the omnidirectional camera picks up an image with about 180 degrees of unnecessary area, which is an inefficient use of the imaging element.